Video Conferencing with Sign language Detection

Video conferencing software has become an essential tool for communicating with one another across long distances. There are several video conferencing software that are utilised for communication all over the world. Huge numbers of people are unable to interact via spoken language and find typical video conferencing solutions difficult to use. Our project intends to solve this problem by creating a user-friendly Video Conferencing App that can identify sign language in real time and provide correct subtitles. Due to a lack of communication skills, deaf and hard of hearing persons confront several obstacles in their everyday lives. The covid epidemic has made traditional ways of communication extremely challenging for these people. Our goal is to bridge the gap by giving them a platform to showcase their skills. © 2022 IEEE.

"Just Not Together": The Experience of Videoconferencing for People with Aphasia during the Covid-19 Pandemic

People with language impairments, such as aphasia, use a range of total communication strategies. These go beyond spoken language to include non-verbal utterances, props and gestures. The uptake of videoconferencing platforms necessitated by the Covid-19 pandemic means that people with aphasia now use these communication strategies online. However, no data exists on the impact of videoconferencing on communication for this population. Working with an aphasia charity that moved its conversation support sessions online, we investigated the experience of communication via a videoconferencing platform. We report a study which investigated this through: 1) observations of online conversation support sessions;2) interviews with speech and language therapists and volunteers;and 3) interviews with people with aphasia. Our findings reveal the unique and creative ways that the charity and its members with aphasia adapted their communication to videoconferencing. We unpack specific, novel challenges relating to total communication via videoconferencing and the related impacts on social and privacy issues. © 2022 ACM.

Automatic Rephrasing of Transcripts-based Action Items

The automated transcription of spoken language, and meetings, in particular, is becoming more widespread as automatic speech recognition systems are becoming more accurate. This trend has significantly accelerated since the outbreak of the COVID-19 pandemic, which led to a major increase in the number of online meetings. However, the transcription of spoken language has not received much attention from the NLP community compared to documents and other forms of written language. In this paper, we study a variation of the summarization problem over the transcription of spoken language: given a transcribed meeting, and an action item (i.e., a commitment or request to perform a task), our goal is to generate a coherent and self-contained rephrasing of the action item. To this end, we compiled a novel dataset of annotated meeting transcripts, including human rephrasing of action items. We use state-of-the-art supervised text generation techniques and establish a strong baseline based on BART and UniLM (two pretrained transformer models). Due to the nature of natural speech, language is often broken and incomplete and the task is shown to be harder than an analogous task over email data. Particularly, we show that the baseline models can be greatly improved once models are provided with additional information. We compare two approaches: one incorporating features extracted by coreference-resolution. Additional annotations are used to train an auxiliary model to detect the relevant context in the text. Based on the systematic human evaluation, our best models exhibit near-human-level rephrasing capability on a constrained subset of the problem. © 2021 Association for Computational Linguistics